crypto: add key object API #24234

tniessen · 2018-11-07T21:48:19Z

Preamble: This is a huge change and I will do my best to help with reviewing it. There might still be dozens of places that need some work, but so far, everything seems to be working nicely. I summarized the motivation behind this change in #15113 (comment). There are also lots of possible discussions around this, e.g. whether key derivation should consume / produce key objects etc.

This commit makes multiple important changes:

A new key object API is introduced. The KeyObject class itself is
not exposed to users, instead, several new APIs can be used to
construct key objects: createSecretKey, createPrivateKey and
createPublicKey. The new API also allows to convert between
different key formats, and even though the API itself is not
compatible to the WebCrypto standard in any way, it makes
interoperability much simpler.
Key objects can be used instead of the raw key material in all
relevant crypto APIs.
The handling of asymmetric keys has been unified and greatly
improved. Node.js now fully supports both PEM-encoded and
DER-encoded public and private keys.
Conversions between buffers and strings have been moved to native
code for sensitive data such as symmetric keys due to security
considerations such as zeroing temporary buffers.
For compatibility with older versions of the crypto API, this
change allows to specify Buffers and strings as the "passphrase"
option when reading or writing an encoded key. Note that this
can result in unexpected behavior if the password contains a
null byte.

cc @nodejs/crypto @nodejs/security-wg @nodejs/security

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

nodejs-github-bot · 2018-11-07T21:48:22Z

@tniessen build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/1531/pipeline

refack

First pass, looks good.
Some guideline nits.

lib/internal/crypto/sig.js

src/node_crypto.cc

doc/api/crypto.md

src/node_crypto.cc

doc/api/crypto.md

TimothyGu · 2018-11-10T01:26:09Z

The new API also allows to convert between different key formats, and even though the API itself is not compatible to the WebCrypto standard in any way, it makes interoperability much simpler.

What are the differences between this new API and the API exposed through web crypto? Could we realistically change this API to be compatible?

tniessen · 2018-11-10T14:06:10Z

What are the differences between this new API and the API exposed through web crypto? Could we realistically change this API to be compatible?

WebCrypto has a very different design. I originally called the new API CryptoKey but changed it to KeyObject to avoid confusion with the WebCrypto CryptoKey class. Some of the key differences:

CryptoKey objects can be exported as raw/SPKI/PKCS#8 only and that is done using a separate function, exportKey, not using an instance method. Our options are much more complex, e.g. to support encrypted keys, and I think it makes sense for this to be an instance method.
The type property is actually compatible to the getType() function of the KeyObject class, and if we decide to implement @addaleax suggestion and turn it into a property, it will match the WebCrypto spec. However, CryptoKey instances have three additional properties that would have a huge impact on the way people can use key objects:
```
readonly attribute boolean extractable;
readonly attribute object algorithm;
readonly attribute object usages;
```
We could implement them, but they don't make sense unless we adopt the whole WebCrypto API in my opinion, and as I said before, I don't think we should do that.
Symmetric CryptoKey objects don't have a property describing the size of the underlying key, and the only way to get the type of an asymmetric key is to parse the algorithm identifier associated with the key.

Rebased, old HEAD was 6c92496a7cf304785162ad5cd37632cea916b22b.

src/node_crypto.cc

src/node_crypto.h

src/node_crypto.cc

sam-github

I really like the direction, I think its a good API. Left some comments, mostly on the API. The C++ is a lot to read through, sorry, I ran out of time. I would prefer this not be experimental. I assume it doesn't intentionally break current APIs, and it also isn't particularly complex in terms of its API, so its hard to see it generating much comment or controversy or need to change. Would be great if it can go into 11.x, get some mileage there, and be stable in 12.xx. I doubt it can make it into 10.x, its a lot of churn.

doc/api/crypto.md

src/node_crypto.cc

doc/api/crypto.md

jasnell · 2018-11-19T17:03:51Z

Definite +1 on this moving forward. Code generally looking good but won't sign off until it's further along.

ChALkeR · 2018-11-23T20:33:55Z

/cc @joepie91

tniessen · 2018-11-25T12:34:49Z

One question has come up multiple times: Should the key object API permit X509 certificates when constructing public keys? Personally, I think it makes more sense to provide an API for certificates that allows to extract the public key as a key object because a certificate is not the same as an asymmetric key, but others might feel differently.

cc @nodejs/crypto

src/node_crypto.h

src/node_crypto.cc

doc/api/crypto.md

This commit makes multiple important changes: 1. A new key object API is introduced. The KeyObject class itself is not exposed to users, instead, several new APIs can be used to construct key objects: createSecretKey, createPrivateKey and createPublicKey. The new API also allows to convert between different key formats, and even though the API itself is not compatible to the WebCrypto standard in any way, it makes interoperability much simpler. 2. Key objects can be used instead of the raw key material in all relevant crypto APIs. 3. The handling of asymmetric keys has been unified and greatly improved. Node.js now fully supports both PEM-encoded and DER-encoded public and private keys. 4. Conversions between buffers and strings have been moved to native code for sensitive data such as symmetric keys due to security considerations such as zeroing temporary buffers. 5. For compatibility with older versions of the crypto API, this change allows to specify Buffers and strings as the "passphrase" option when reading or writing an encoded key. Note that this can result in unexpected behavior if the password contains a null byte.

This commit makes multiple important changes: 1. A new key object API is introduced. The KeyObject class itself is not exposed to users, instead, several new APIs can be used to construct key objects: createSecretKey, createPrivateKey and createPublicKey. The new API also allows to convert between different key formats, and even though the API itself is not compatible to the WebCrypto standard in any way, it makes interoperability much simpler. 2. Key objects can be used instead of the raw key material in all relevant crypto APIs. 3. The handling of asymmetric keys has been unified and greatly improved. Node.js now fully supports both PEM-encoded and DER-encoded public and private keys. 4. Conversions between buffers and strings have been moved to native code for sensitive data such as symmetric keys due to security considerations such as zeroing temporary buffers. 5. For compatibility with older versions of the crypto API, this change allows to specify Buffers and strings as the "passphrase" option when reading or writing an encoded key. Note that this can result in unexpected behavior if the password contains a null byte. PR-URL: #24234 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

PR-URL: #24234 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

Notable Changes: * cli: - add --max-http-header-size flag (cjihrig) #24811 * crypto: - always accept certificates as public keys (Tobias Nießen) #24234 - add key object API (Tobias Nießen) [#24234](#24234) - update root certificates (Sam Roberts) #25113 * deps: - upgrade to libuv 1.24.1 (cjihrig) #25078 - upgrade npm to 6.5.0 (Audrey Eschright) #24734 * http: - add maxHeaderSize property (cjihrig) #24860 PR-URL: #25175

This commit makes multiple important changes: 1. A new key object API is introduced. The KeyObject class itself is not exposed to users, instead, several new APIs can be used to construct key objects: createSecretKey, createPrivateKey and createPublicKey. The new API also allows to convert between different key formats, and even though the API itself is not compatible to the WebCrypto standard in any way, it makes interoperability much simpler. 2. Key objects can be used instead of the raw key material in all relevant crypto APIs. 3. The handling of asymmetric keys has been unified and greatly improved. Node.js now fully supports both PEM-encoded and DER-encoded public and private keys. 4. Conversions between buffers and strings have been moved to native code for sensitive data such as symmetric keys due to security considerations such as zeroing temporary buffers. 5. For compatibility with older versions of the crypto API, this change allows to specify Buffers and strings as the "passphrase" option when reading or writing an encoded key. Note that this can result in unexpected behavior if the password contains a null byte. PR-URL: #24234 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

PR-URL: #24234 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

Notable Changes: * cli: - add --max-http-header-size flag (cjihrig) #24811 * crypto: - always accept certificates as public keys (Tobias Nießen) #24234 - add key object API (Tobias Nießen) [#24234](#24234) - update root certificates (Sam Roberts) #25113 * deps: - upgrade to libuv 1.24.1 (cjihrig) #25078 - upgrade npm to 6.5.0 (Audrey Eschright) #24734 * http: - add maxHeaderSize property (cjihrig) #24860 PR-URL: #25175

During the time between nodejs#24234 being opened and it landing, a V8 update occurred that deprecated several APIs. This commit fixes the following compiler warnings: ../src/node_crypto.cc:3342:11: warning: 'Set' is deprecated: Use maybe version ../src/node_crypto.cc:3345:13: warning: 'GetFunction' is deprecated: Use maybe version PR-URL: nodejs#25205 Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>

Notable Changes: * cli: - add --max-http-header-size flag (cjihrig) #24811 * crypto: - always accept certificates as public keys (Tobias Nießen) #24234 - add key object API (Tobias Nießen) [#24234](#24234) - update root certificates (Sam Roberts) #25113 * deps: - upgrade to libuv 1.24.1 (cjihrig) #25078 - upgrade npm to 6.5.0 (Audrey Eschright) #24734 * http: - add maxHeaderSize property (cjihrig) #24860 PR-URL: #25175

During the time between #24234 being opened and it landing, a V8 update occurred that deprecated several APIs. This commit fixes the following compiler warnings: ../src/node_crypto.cc:3342:11: warning: 'Set' is deprecated: Use maybe version ../src/node_crypto.cc:3345:13: warning: 'GetFunction' is deprecated: Use maybe version PR-URL: #25205 Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>

This commit makes multiple important changes: 1. A new key object API is introduced. The KeyObject class itself is not exposed to users, instead, several new APIs can be used to construct key objects: createSecretKey, createPrivateKey and createPublicKey. The new API also allows to convert between different key formats, and even though the API itself is not compatible to the WebCrypto standard in any way, it makes interoperability much simpler. 2. Key objects can be used instead of the raw key material in all relevant crypto APIs. 3. The handling of asymmetric keys has been unified and greatly improved. Node.js now fully supports both PEM-encoded and DER-encoded public and private keys. 4. Conversions between buffers and strings have been moved to native code for sensitive data such as symmetric keys due to security considerations such as zeroing temporary buffers. 5. For compatibility with older versions of the crypto API, this change allows to specify Buffers and strings as the "passphrase" option when reading or writing an encoded key. Note that this can result in unexpected behavior if the password contains a null byte. PR-URL: nodejs#24234 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

PR-URL: nodejs#24234 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

During the time between nodejs#24234 being opened and it landing, a V8 update occurred that deprecated several APIs. This commit fixes the following compiler warnings: ../src/node_crypto.cc:3342:11: warning: 'Set' is deprecated: Use maybe version ../src/node_crypto.cc:3345:13: warning: 'GetFunction' is deprecated: Use maybe version PR-URL: nodejs#25205 Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>

Notable Changes: * cli: - add --max-http-header-size flag (cjihrig) nodejs#24811 * crypto: - always accept certificates as public keys (Tobias Nießen) nodejs#24234 - add key object API (Tobias Nießen) [nodejs#24234](nodejs#24234) - update root certificates (Sam Roberts) nodejs#25113 * deps: - upgrade to libuv 1.24.1 (cjihrig) nodejs#25078 - upgrade npm to 6.5.0 (Audrey Eschright) nodejs#24734 * http: - add maxHeaderSize property (cjihrig) nodejs#24860 PR-URL: nodejs#25175

Expose the size of asymetric keys of crypto key object from the crypto module added in v11.6.0 (nodejs#24234)

Expose the size of asymetric keys of crypto key object from the crypto module added in v11.6.0. PR-URL: nodejs#26387 Refs: nodejs#24234 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

benbucksch · 2020-05-20T12:03:21Z

The new API introduced here is extremely ambiguous and confusing.
We have a "KeyObject" and a "key object", which are not the same type.
We have KeyObject type, key object, key parameter, and key property. All different.
We have key.key.key. Not kidding.

tniessen added the semver-minor PRs that contain new features and should be released in the next minor version. label Nov 7, 2018

nodejs-github-bot added the lib / src Issues and PRs related to general changes in the lib or src directory. label Nov 7, 2018

tniessen added crypto Issues and PRs related to the crypto subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. labels Nov 7, 2018

tniessen mentioned this pull request Nov 7, 2018

Support parsing pem key only once for crypto api #15113

Closed

refack reviewed Nov 8, 2018

View reviewed changes